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Abstract. Graphs are often used to 
represent mathematical functions, to 
illustrate data from social and natural sci- 
ences, or to specify scientific theories. With 
increasing emphasis on the development 
of scientific research skills, the work with 
graphs and data interpretation are gain- 
ing in importance. The research involved 
an eye-tracking experiment conducted 

to evaluate student work with graphs 

in physics. Eye-movement data were 
recorded using the GazePoint eye-tracker. 
A total of 40 third-year grammar school 
students participated in the research. 
These students were allocated into three 
groups by a physics teacher. These groups 
were called PLUS, AVERAGE and MINUS. 
The PLUS group showed excellent results 

in education and included gifted physics 
students. The MINUS group was com- 
posed of the opposite end of this cognitive 
spectrum, whose members made the most 
mistakes in graph reading. The aim of the 
experiment was to find the differences 
between students allocated to these three 
groups and to evaluate whether the al- 
location based on the teacher's experience, 
long-term observations and the students’ 
previous achievements was sufficient. The 
results showed that students from all three 
groups had problems with reading graphs 
in physics. According to the eye-movement 
data, several students who had been incor- 
rectly assigned to groups were identified. 
Keywords: education in physics, gifted 
children, graph, eye-tracking, experimental 
study. 
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Introduction 


Education at school in the Czech Republic follows a Framework Edu- 
cational Programme (FEP) (Jerabek et al., 2016). In the context of this pro- 
gramme, each school, including grammar schools, creates its own School 
Educational Programmes (SEP), which may include studies in physics. The 
content of studies in physics is given by the curriculum, while the extent and 
arrangement of education, to a certain extent, is the teacher’s responsibility. 
Theoretical schoolwork accompanies experiments, and solving examples 
and tasks is followed by practical exercises. The main aim of studies in phys- 
ics at secondary school is to present physics to students as an exact science 
through suitable forms and using appropriate methods to achieve knowl- 
edge, understanding, skill and habits for future study and practice. Gram- 
mar school education of physics is preceded by primary school education. 
Students acquire knowledge step by step, usually through observation and 
simple measurements, and thereby their relationship to the subject begins. A 
grammar school course of four years, sometimes three, is more difficult than 
the primary school course, and for some students, it is the final education in 
physics offered by schools. Grammar schools in the Czech Republic are seen 
as schools which prepare their graduates for university studies, not entry into 
practice. Physics taught at Grammar schools includes studies of mechanics 
(kinematics and dynamics), molecular and thermal physics, electricity and 
magnetism, optics, oscillation and waves. Students are also introduced to 
the basics of astronomy and the special theory of relativity. They encounter 
graphs in all of these disciplines. Graphs “assault” students even in mathemat- 
ics, chemistry, biology, and other subjects, not only natural sciences. 

Science subjects are generally less popular with students. However, this 
statement is not necessarily always valid. For example, concerning students 
interested in physics, Poschl (2007) mentioned that students who perceive 
physics more as a scientific field have a more positive attitude towards it than 
those who perceive it simply as a subject. 

Coleman and Cross (2005) offered 12 principles for teachers for the basis 
of their decisions in the classroom. Applying all the principles is a complicated 
task since the teacher must make many decisions according to situation. 
Teachers rarely use one principle at a time, and they make more than one 
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decision during an active teaching period. A“teaching system’ is a much broader concept than a“teaching method’, 
which is acommon description in the literature (Coleman & Cross, 2005). 


Gifted Students 


The topic of talented and gifted students is currently important in the Czech Republic. The Ministry of Educa- 
tion, Youth, and Sports of the Czech Republic consider the support and development of gifted students one of its 
main priorities, especially in the fields of natural sciences and technology. An important official step in supporting 
talented students was Decree No. 27/2016 Coll. and testifies to the social need of supporting gifted students in the 
Czech education system (MSMT, 2019). 

Gifted children and students with special educational needs in the Czech Republic are currently addressed 
in the context of an inclusion process. Populations with special needs, which is the area gifted children fall under, 
are included in the majority education system while taking into account the educational needs of all children. 

In the inclusion process, children meet each other in class, and the teacher determines how to manage the 
situation. If the exact diagnoses of the children are available, this process is done by psychologists and special edu- 
cators in a special environment. Teachers may also designate children for diagnosis. Through teacher training, we 
can infer that allocating a child to a group relies on excellent observation of the child’s behaviour and performance 
and that the main aim is to establish the best possible educational strategies in order to identify and eventually 
diagnose a child. Diversification of children into groups according to their abilities is always demanding and sensitive. 

Identifying gifted children is a challenging and perplexing problem for people interested in promoting the 
development of high ability (Coleman & Cross, 2005). It is interesting when children are identified; they implicitly 
have a higher probability of behaving like gifted adults than other children do, although this is an assumption, 
because the data relating to childhood and adult performance are inconsistent (McClelland et al., 1958). Coleman 
and Cross (2005) stated that it is easier to make statements about groups than individuals. 

The process of identifying gifted individuals is unique (Clark, 1988; Tuttle Jr, 1988). Davis and Rimms (1989) 
suggested four stages of identification. This distribution is accepted by other authors; for example, Renzulli (1990). 
Standardized tests are still the most frequently mentioned technique for measuring abilities (Alvino et al., 1981). 


Reading Graphs 


Graphs included in common software (e.g. Microsoft Excel) can be used by scientists or politicians and can also 
express the results of research conducted by companies or the media. The ability to read graphs and understand them 
should be a basic skill of every person. After all, most primary school and secondary school graduates need this skill 
for their vocations or further studies, even if their studies were not in natural sciences. In physics, graphs are used asa 
visual image of the mutual dependencies between physical quantities. Students already encounter graphs in physics 
at primary schools. At grammar schools, graphs are commonly included in physics studies, and students should not 
have any problems reading them. Graduates leave secondary schools for university studies, and the ability to read 
graphs is usually taken for granted. The population in the modern world is awash with data. But as anyone who has 
ever opened up a spreadsheet filled with seemingly infinite lines of data knows, numbers aren't enough: people need 
to know how to make those numbers talk (Page, 2018). Each of these areas contains more specific issues that cause 
problems for students. Glazer’s (2011) study states that with the growing emphasis on developing scientific inquiry 
skills, the display and interpretation of data are becoming increasingly important. In fact, Competence in graph inter- 
pretation is essential to understanding the today’s world and being scientifically literate. However, interpreting graphs 
is acomplex and challenging activity. Competence in interpreting graphs is affected by many factors, including the 
characteristics of the graph, its content, and the reader’s prior knowledge (Glazer, 2011). 

The subject of physics has difficult aspects that students struggle with, and reading graphs is indisputably 
one of them (Leinhardt et al., 1990): 

e Students often perceive graphs as pictures or sketches of a given situation, not as abstract mathemati- 
cal representations. 

e Students often do not know whether the required information is acquired from the direction or the slope. 

e Students have problems with determining direction (they often count it as a value from a single point). 

e Students do not recognize whether an interval or point value is required and tend to interpret from a 
pointwise perspective. 
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Why is it appropriate to select only graphs used in physics for this research? Reading graphs of functions is 
more likely encountered in mathematics, but mathematics also serves a numerical apparatus for physics, and no 
choice exists to work without graphs or without the ability to use them, especially not if teachers want to give 
students a complete idea of the phenomena and processes described and interpreted in physics. Experiments are 
an inseparable part of most scientific research, and graphs are very important in their evaluation, because a graph 
displaying a physical phenomenon, at first glance, enables a trend of development not recognizable in a table of 
the same values to be seen (Beichner, 1994). According to some external studies published in the past 20 years or 
so, students of all ages have demonstrated problems in reading and understanding graphs. It is therefore invalu- 
able to discover how Czech students at grammar schools process work with graphs. 

The results of some research conducted in the field of mathematics is inspiration. This research has confirmed 
the hypothesis that using the formal notation characteristics of a programming language for presenting algorithms is 
often a practical difficulty in the process of solving even simple tasks. The research has also shown that eye-tracking 
technology can be used to optimize the educational process of learning programming (Andrzejewska et al., 2016). 


Eye-Tracking in Education 


Eye-tracking technology has been used to allocate (not identify) gifted students. According to Shayan et 
al. (2017), the special contribution of eye-tracking to educational research has been in combining the data of 
students’ physical movements and verbal utterances. There are two basic approaches to using eye-tracking in 
education and didactics: evaluation of created materials and user evaluation. Jarodzka et al. (2017) presented 
the theoretical framework and research agenda for this area. Eye-tracking has also been studied by Bolden et al. 
(2015), Skrabankova and Trnova (2015) and Rosengrant et al. (2009) to examine how students solved problems in 
physics and mathematics. 

Eye-tracking can also be used directly during class time. The first option would be equipping the teacher with 
eye-tracking glasses and assessing his or her responses to students. An alternative would be watching a video of 
the lesson. Such a study was conducted by Yamamoto & Imai-Matsumura (2013). 

The differences between expert and novice teachers were analysed using the eye-tracking method in a study 
by Wolff et al. (2017). 

Eye-tracking has also been previously used in a study with gifted children. Eye-tracking glasses PupilLabs were 
used in a study to examine creativity in mathematics (Schindler et al., 2016). Mathematical stimuli were also used 
in a study (Sajka & Rosiek, 2015) comparing the differences between the problem-solving processes of mathemati- 
cally gifted students and underperforming students. In a study (Choi et al., 2012), eye-movement patterns were 
analysed to find the differences between average and gifted students. An overview of eye-tracking in education 
was presented in the book by Was et al. (2016). 


Eye-Tracking and Graphs 


Shah and Hoeffner (2002) stated that graphs are commonly used to represent mathematical functions, show 
data from social and natural sciences, and specify scientific theories in textbooks and other print media in and out 
of the classroom. 

With the growing emphasis on the development of scientific inquiry skills, work with graphs and the interpre- 
tation of data is becoming increasingly important (Glazer, 2011). Graphs are important for understanding scientific 
data (Shah & Hoeffner, 2002). 

Graph interpretation is a fundamental skill that is essential for all students in their everyday lives. According to 
Glazer (2011), little research in graph interpretation in science (as opposed to mathematics) has been conducted. 
This statement is consistent with Shah and Hoeffner (2002), who affirmed that graphs are important in understand- 
ing scientific data. 

Strobel et al. (2016) mentioned that elementary graph reading skills can be observed as early as preschool and 
middle school (Curcio, 1987) and that these skills develop over time (Lowrie & Diezmann, 2007). Graph literacy, i.e. 
the ability to understand graphically presented information, may be measured, for example, in the scale developed 
by Galesic and Garcia-Retamero (2011). However, relatively little information exists about how people extract infor- 
mation from graphs and how graphical design defines or‘nudges’and biases decision making (Vila & Gomez, 2016). 
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Previous decades saw the development of many taxonomies comprising highly similar concepts (for example, 
Bertin (1983); Curcio (1987); Tan and Benbasat (1990) or Wainer (1992)). 

Eye-tracking has already been used in several studies concerning graphs. Goldberg and Helfman (2011) intro- 
duced the potential of eye-tracking in a study comparing radial and linear graphs. The results showed that linear 
graphs could better support the initial, dimension-finding stage than radial graphs. A similar comparison of linear 
and radial graphs (marked as polyline and star glyphs) was conducted in another eye-tracking study by Opach et 
al. (2017). The results showed that each type of graph was suitable for different types of tasks. 

Atkins and McNeal (2018) conducted an eye-tracking study exploring graphs in the context of climate change. 
Van der Linden et al. (2014) stated that graphs are the most effective way of communicating the climate consensus 
of researchers. Fifty-eight graduate and undergraduate students were involved in the study, which presented four 
open-ended tasks testing fact extraction and data interpolation. The results showed that the time spent on differ- 
ent graph features varied between the graduate and undergraduate student groups. 

Extracting business information from graphs was examined in a study by Vila and Gomez (2016), who applied 
the methodology of experimental economics. The authors compared graphs which had numerical information 
to graphs which had none. The results showed that the inclusion of values appeared to have no impact on the 
accuracy of the answers, though increased the time required to extract the information from the graph. 

Kim et al. (2014) and Kim and Wiseheart (2017) analysed graph reading by dyslexic and non-dyslexic users. 
Moreno-Esteva et al. (2017) presented a mathematical and computational analysis, partially based on eye-tracking. 

The research was focused on the analysis of the use of graphs in physics. Students’ comprehension of graphs 
has been researched in many studies of education in physics. The paper by Susac et al. (2018) contained a com- 
prehensive review of over 30 studies which examined the use of kinematics graphs to explore student graphing 
abilities. Few of these studies, however, involved eye-tracking. 

Susac et al. (2018) compared physics and psychology students’ comprehension of graphs. The tasks involved 
interpreting graph slope and comprehending the area beneath it. The results indicated that area under a graph 
is a difficult concept and that more attention should be given to it in physics classes. Tai et al. (2006) used eye- 
tracking to identify problem-solving behaviours in a group of individuals with different levels of expertise in biol- 
ogy, chemistry, and physics. 

Kozhevnikov et al. (2007) presented three studies on the relationship between spatial ability and kinematics 
problem solving. The third study employed eye-tracking analysis, and the authors examined whether students 
made eye-movements in the correct direction. To test spatial ability, Paper Folding, Form Board, Cube Comparison, 
and Card Rotation tasks were conducted. The authors argued that spatial ability might be important for visualizing 
invisible phenomena and processes such as electric or magnetic field lines or electric current. 

Madsen et al. (2012) analysed how visual attention differed between those who either correctly or incorrectly 
answered introductory physics problems. The authors found that top-down processes played an important role 
in guiding visual attention. Top-down processes are based on the viewer's previous knowledge, task goals, and 
expectations. Viiri et al. (2017) had a similar aim and explored the differences between students who responded 
correctly and those who responded incorrectly. Heat maps showed that the students who responded incorrectly 
more frequently viewed the irrelevant parts of a task. Kekule (2014) performed a similar study dealing with kine- 
matics graphs. 

Goldberg and Helfman (2011) described three steps needed to interpret and understand graphs: (1) examining 
values, (2) comparing values, and (3) appreciating higher level patterns, such as trends and distributions. Bertin 
(1983) identified three major components of graph reading. The first component involves viewers encoding the 
visual array and identifying the important visual features. In the second component, viewers relate the graph’s 
visual features to the conceptual relationships represented by those features. The third component is the process 
of comprehending a graph, in which viewers determine the referent of the quantified concepts and associate 
those referents to encoded functions (Shah & Hoeffner, 2002). Shah and Hoeffner (2002) highlighted that these 
three processes imply that three factors play an important role in determining a viewer's interpretation of data: 
the characteristics of the visual display (bar or line graph), knowledge of graphs (graph schemas), and content 
(e.g., age vs height, time vs distance). 

The present research involved tasks to identify graphs which described certain physical phenomena. The 
character of the tasks ensured that the third factor would be crucial: participants had to recognize the content, i.e. 
the physical phenomena, and identify it in the graphs. 
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The aim of the research was to understand how students read graphs and analyse the differences of graph 
reading according to respondent's groups. 

Selecting the research group was consulted with an experienced physics teacher, who was a key person in 
the experiment. The teacher allocated students into three groups according to cognitive performance. The groups 
were designated PLUS, AVERAGE and MINUS. The teacher allocated students to these three groups based on 
experience with the students, long-term observations and the students’ previous achievements. The PLUS group 
showed excellent results in education and was expected to include generally talented students in physics and 
natural sciences. This means that members of this group should not have had any problems reading graphs. The 
MINUS group formed the opposite end of this cognitive spectrum, and its members were expected to make the 
most mistakes in reading graphs. The AVERAGE group separated both extreme groups and should have contained 
participants with average results. 

After allocating the students, the research examined two basic, complementary levels which crossed over 
in the results. In the first level, the aim was to understand how students from individual groups proceeded when 
they read graphs and how they were able to solve related tasks displayed on a screen. The aim of the second level 
was to detect the hidden potential of students present in the research sample, regardless of their particular group. 
According to Betts and Neihart (1988), such individuals might be expected in groups of students/participants. 
These designated “underground talent” students (even young students found at primary schools) could not be 
detected by the teacher from their educational behaviour, which may be intentionally downplayed (Neihart, 2011). 
Therefore, it is important to study the results of their work examined in this research. 


Three research questions were posed: 
1. Is the statement that students read graphs with difficulty valid? 
2. Can similar segments be found in the acquired data which shows clusters of similar attitudes be- 
tween participant groups in solving the given tasks? 
3. Can students who were incorrectly allocated to a group by the teacher be revealed through the ac- 
quired data? 


An eye-tracking experiment was created to explore these research questions. The experiment’s aim was to 
highlight the diversity in students’ cognitive abilities in physics. Graphs demonstrating mechanics (kinematics), 
oscillation and waves, and molecular physics phenomena were used as source images. These graphs were selected 
for their explanatory character. 

The PLUS group of participants was selected to formulate the hypotheses. This group represented students 
with the highest potential for learning in physics: 

e H.: Group PLUS needs the least amount of time to solve the given tasks. 

¢ H,: Group PLUS makes fewer mistakes in task solving. 

¢ —H,:The correctness of answers is proportional to the number of fixations in an AOI (Area of Interest). 

¢ H,: Students who were mistakenly not assigned to the PLUS group can be found by analysing the 
acquired data. 

¢ H,:The teacher's allocation to groups based on cognitive abilities is not always correct. 


Research Methodology 
Experiment Design 


The eye-tracking experiment was conducted at Gymndzium Novy Jicin in two sessions at the beginning of 
2019 (January 29 and January 31). Three eye-trackers were placed in the classrooms, and the participants came one 
at a time to the testing venue. Trained technicians introduced the principle of the eye-tracker and explained the 
organization of the experiment. After calibration, a fixation cross appeared for 600 ms to unify the starting position 
of eye-movement trajectories. A stimulus was then displayed for an indefinite time. The task required an answer 
to the question in the upper part of the stimuli. The participants had to select one of the graphs and announce 
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the answer aloud. The answer was marked into a pre-prepared list by a technician. If a participant resoonded that 
they did not know the answer, an “X” was marked into the list instead of the answer. The scheme of the procedure 
is displayed in Figure 1. 





Figure 1 
Scheme of the procedure 
600 ms Unlimited time Aloud 
Introduction Calibration Pe | stimtus |» [sent tome fe 
7 times 
Equipment 


A GazePoint 3 eye-tracker with a recording frequency of 60 Hz was used to collect data in this experiment. 
The eye-tracker was connected to a notebook and placed under a monitor with a resolution of 1280 x 1024 pixels. 
The device was an inexpensive eye-tracker similar to TheEyeTribe tracker or Tobii EyeX. The accuracy and preci- 
sion of these devices have been tested in studies by Dalmaijer (2014), Popelka et al. (2016), and Ooms et al. (2015). 
The results of these studies showed that this kind of device can be applied to scientific use. The GazePoint eye- 
tracker was evaluated only in a study by Janthanasub and Meesad (2015), however it achieved similar results to 
the TheEyeTribe and DIY eye-trackers. 


Stimuli and Tasks 


Basic secondary school physics graphs generally encountered by students during their studies were displayed 
for reading. The graphs showed principles in mechanics (kinematics), oscillations and waves, and molecular physics. 
These areas covered secondary school physics over three years of study (mechanics/kinematics — 1*t year, molecu- 
lar and thermal physics — 2"¢ year, oscillations and waves — 3'¢ year). Under the teacher's guidance, students can 
discover their own as well as recommended strategies for reading graphs and learn to appreciate the clarity and 
the content of information presented in graphs. During their studies, students encounter many types of these 
graphs, which represent basic graphical interpretations of the physical processes and phenomena and knowledge 
of physical areas they study. It is clear that physics teachers work with these graphs in education and students 
should know them. Moreover, understanding the principles of reading graphs is not based on memorization but 
comprehension. It is not relevant to refer to distant learning or short-term memory. If comprehension of graph 
principles exists, the prerequisite of some ability independent of time also exists. 

Stimuli were presented in a fixed order. The tasks formed a part of the stimuli, and their English translation is 
given below. The overview of all tasks and stimuli is displayed in Figure 2. 


¢ —1A/Which of the graphs represents lines of uniform motion at initial track s,? 

e =1B/ Which of the graphs represents a track dependent on uniformly accelerated motion in time? 
e = 2A/ Which of the pV diagrams has an isochore? 

e =2B/ Which of the pV diagrams has an isotherm? 

e =2C/ Which of the pV diagrams has an isobar? 

e 3A/Which image represents forced oscillation? 

e 3B/ Which image represents free oscillation of the oscillator? 
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Stimuli used in the experiment 


Ktery z grafd predstavuje graf drdhy rovnomérného pohybu pii pocatetni draze so? Ktery z graf pfedstavuje graf drdhy rovnomérné zrychleného pohybu na ¢ase? 





Na kterém z pV diagramd je znazornéna izochora? Na kterém z pV diagramd je znazornéna izoterma? Na kterém z pV diagramd je znazornéna izobara? 


©, ©, AR ©, AR ©, 
: lag K 2A Hy, $k 2B Ua sk 


Které zobrazeni predstavuje nucené kmitani? Které zobrazeni pfedstavuje viastni kmitani oscilatoru? 
SS : —* 
| | \ j \ / — 
@: Cera! Y 
| A VU 
| lita 
| I\Nnan. t 














Participants 


The research was conducted on a sample of 52 high school (Gymndzium Novy Jicin) students from the third 
grade. The students were 16 years old and formed a basic research group. From this primary group, 12 students 
were removed due to technical problems with the eye-tracker or insufficient calibration results. The experiment 
was therefore conducted on 40 participants (9 males and 31 females). 

The participants were allocated into three groups, PLUS, AVERAGE, and MINUS, according to the opinion of 
their physics teacher. The results of students’ previous studies were used in combination with the teacher’s recom- 
mendations as a “distribution parameter”. Professional diagnostics is more complicated, and not all students are 
approved in this manner. Few students are ever identified, mainly because of improper allocation and a certain 
unwillingness to participate in diagnostics. 

The teacher allocated students to these three groups based on experience with the students, long-term ob- 
servations and the students’ previous achievements. The PLUS group showed excellent results in education and 
was expected to include generally talented students in physics and natural sciences. This means that members of 
this group should not have had any problems reading graphs. The MINUS group formed the opposite end of this 
cognitive spectrum, and its members were expected to make the most mistakes reading graphs. The AVERAGE 
group separated both extreme groups and should have contained students with average results. 

The PLUS group contained 11 participants. The AVERAGE group was the largest, with a total of 24 participants. 
The remaining five participants were weak in physics and allocated to the MINUS group. The participants were 
designated PO1-—P40. 
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Data Pre-Processing 


The data recorded by the eye-tracker required validation before it could be analysed. The recording was 
conducted in the classroom, and students had only one previous experience with eye-tracking testing. Some of 
them, therefore, may have been nervous, which may affect the quality of recorded data. 

In the first step of data pre-processing, recorded data were exported from the GazePoint Analysis environ- 
ment into the open-source application OGAMA (Vo&kuhler et al., 2008) providing more advanced functionality. In 
OGAMA, the fixation detection algorithm (I-DT) thresholds were set to 20 pixels (distance between points) and 5 
(minimum number of samples). The optimal fixation detection algorithm has been described by (Popelka, 2014) in 
more detail. In the next step, the eye-movement trajectories of all participants were checked visually, and problem- 
atic examples were removed from the dataset. Data loss information and off-screen samples were then exported 
from OGAMA. The value of data loss represents the ratio of samples with coordinates 0;0 (upper-left corner of the 
stimulus). The second value shows how many samples were recorded with coordinates off the screen. Both values 
were summarized, and participants with a sum (data loss + off-screen) greater than 15% were removed from the 
experiment. As mentioned above, the experiment comprised a total of 40 participants with valid data. 


Methods of Analysis 


Several methods of eye-tracking data analyses were applied in the research. Data were statistically analysed 
in RStudio using the Kruskal-Wallis post hoc Nemenyi test, which is a non-parametric variant of ANOVA. Statisti- 
cally significant differences (p < .05) were indicated in the boxplots. The Trial Duration metric was selected for the 
analysis. A longer time required to solve a task indicated a task with greater complexity. 

Data were visualized using sequence charts, which represented participants’ eye-movement data with coloured 
bars. The colour of each cell in a bar represented one fixation in a particular Area of Interest (AOI). The sequence 
chart, therefore, showed the order of visited AOls, how much time participants spent looking at them, and whether 
they repeatedly looked at certain AOls. 

Since neither OGAMA nor GazePoint Analysis permits visualizations with sequence charts, the charts were 
created manually in MS Excel using the PART and conditional formatting functions. Fixations in the indicated AOls 
were represented with coloured rectangles. 

The final analytical method was a scanpath comparison using the ScanGraph tool. This tool was developed 
to identify differences in the stimulus reading strategies of different groups of participants (Dolezalova & Popelka, 
2016; Popelka et al., 2018). ScanGraph read the order of visited AOls (the same used to visualize the sequence 
charts) and calculated the similarity of these strings by employing three different algorithms: Levenshtein distance, 
the Needleman—Wunsch algorithm and Damerau-Levenshtein distance. Individual participants were visualized as 
nodes on a graph, and ScanGraph searched for cliques in this graph, i.e. groups of participants who were similar 
to each other at least to a certain degree. The tool can be used to determine whether the stimulus was read differ- 
ently, for example, by men or women or by experts or novices. 


Research Results 
Accuracy of Answers 


The first step in the data analysis examined the accuracy of answers. Participants resoonded aloud, and their 
answers were recorded into a form by technicians. The data and correct answers are summarized in Table 1. The 
table shows that the majority of answers were incorrect. Incorrect answers are indicated in the figure in grey. Only 
39% (110 out of 280) of the answers were correct. It could be assumed that the easiest tasks were tasks 3A and 3B. 
However, the higher density of correct answers was a result of participants only selecting between two options 
in task 3A, while the other tasks had four options (A, B, C and D). In task 3B, both options were correct (A and B). 
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Table 1 


Correct answers in the experiment 


ID Sex Group 1A 1B 2A 2B 2C 3A 3B Correct Incorrect 
P05 F Minus B B C D C B B 1 6 
P16 F Minus B C B C D A B 5 2 
P20 F Minus B B A D B A A 3 4 
Pao F Minus B B B C A B A 2 5 
P40 F Minus B D C C C A A 3 4 
P01 F Average D D C C A A A 3 4 
P03 F Average B A C B D B A 2 5 
P04 M Average D C 6 D D A B 4 3 
P06 F Average B D B C C A B 3 4 
P07 F Average D B A D B B A Z 5 
P09 F Average B B C C A A B 2 4 
P10 F Average B B D C B A B 3 4 
P11 F Average D B C B A A B 2 5 
P15 F Average D C B C D A A 5 Z 
P18 F Average B D B C A A X 2 5 
P19 F Average D C D A D A B 4 3 
P21 M Average D C A D A B B 3 4 
P23 F Average B D A D B A B 3 4 
P25 F Average D B B C D A B 4 3 
P26 F Average B C 6 D B A B 3 4 
P29 M Average D C A D B B A 3 4 
P30 F Average B B C B D A A 3 4 
P31 F Average D B A D C A A 3 4 
P32 F Average B D A C D B A 4 3 
P33 F Average D A B A D B B 2 5 
P34 F Average D C B A D B A 3 4 
P36 F Average D D D A C A A 2 5 
Po? F Average B B C C B A A d 4 
P38 M Average B A X X X A B 2 5 
P02 F Plus D D C C C B A 2 5 
P08 F Plus D B D D A A A 2 5 
P12 F Plus D B D A B A B 2 5 
P13 M Plus A C A D A B B 4 3 
P14 M Plus B C X A D B X 2 5 
P17 M Plus B A A D D B B 3 4 
P22 F Plus B C B A D A C 3 4 
P24 M Plus B A A B X B D 1 6 
P27 F Plus B D D A B A B 2 5 
P28 F Plus B C B D A A B 3 4 
P39 M Plus B A B D C B A 1 6 

Correct 1 12 10 13 13 29 36 
Incorrect 39 28 30 27 27 15 4 
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The best results were obtained by the AVERAGE group, with an accuracy of 42.3%. Surprisingly, the PLUS 
group obtained the worst results, with an accuracy of only 32.5%. It should be noted that the data may have been 
affected by participants P24 and P39, who solved only one task correctly. The accuracy of the MINUS group was 
40%. This group contained only five participants, which may also have affected the results. This group also con- 
tained a participant (P05), who solved only one task correctly. Data were statistically analysed, but no statistically 
significant differences between the groups were found. 


Stimuli as a Whole 


At the beginning of the statistical analysis, data recorded for all stimuli were analysed as a whole. In the first 
step, the differences between the groups were not analysed. The values for Trial Duration for all stimuli are given 
in Figure 3. Lines in the upper part of the boxplot represent pairs where statistically significant differences were 
found using the Kruskal-Wallis post hoc Nemenyi test. The boxplot shows that the majority of statistically signifi- 
cant differences were between different task groups (1, 2 and 3). Inter-group differences were found only in the 
case of pairs 2A-2C, while a higher Trial Duration value (longer time) was observed in the first task. This is logical, 
because the participants had to familiarize themselves with the task at the beginning (A), and their work was then 
faster, as the tasks in each group were similar (B and C). Task 3 indicated the contrary, as solving task 3B took more 
time than solving 3A. Though in this case, it was because task 3B contained four options and task 3A only two. 


Figure 3 
Trial Duration for all tasks. Statistically significant differences are indicated with lines in the upper section 
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The eye-tracking metrics of the Fixation Count and Scanpath Length had a similar distribution of values to 
Trial Duration. The Trial Duration values for all three groups are shown in Figure 4. The time required to solve the 
task varied between the groups. The hypothesis that the PLUS group would need less time to solve the task than 
the MINUS group was only corroborated in tasks 2C and 3A. Task 1A demonstrated the contrary, i.e. that the MINUS 
group needed the least amount of time of all three groups. The results were not statistically significant. 


307 


https://doi.org/10.33225/jbse/20.19.298 LTT 


308 


Journal of Baltic Science Education, Vol. 19, No. 2, 2020 


STUDENTS’ ABILITY TO WORK WITH GRAPHS IN PHYSICS STUDIES RELATED TO THREE ISSN 1648-3898 


/Print/ 
TYPIGAL STUDENT GROUPS 
(P. 298-316) ISSN 2538-71 38 /Online/ 
Figure 4 


Trial Duration for all three participants groups in all tasks 
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Areas of Interest 
In the next step, Areas of Interest (AOIs) were marked in the stimuli on each task and graph. The distribution 
of AOls is illustrated in Figure 5. The circles with numbers show the number of fixations recorded at each AOI. Grey 


rectangles indicate correct answers. 


Figure 5 
Distribution of fixations in AOIs 
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In many cases, the AOI with a low number of or even the fewest recorded fixations corresponded to the cor- 
rect answer. Examples are task 1B and 2B. These two tasks recorded the fewest fixations in AOI“C’, which was the 
correct answer in both cases. In task 3B, two options were correct (A and B); and both of these were fixated on 
extensively. Other answers were fixated on only marginally. 

In the next step, the data were explored with respect to differences between groups. The Fixation Count for 
each AOI and group was calculated. The averages of these values are given in Figure 6. The PLUS group demon- 
strated the most fixations while reading in almost all tasks (except for task 3B). The difference was not statistically 
significant, but in task 2A, the difference between the MINUS and AVERAGE groups indicated a significance of p 
= .086. The only significant difference in the entire dataset was found in task 2B and AOI“B”. In this task, the PLUS 
group demonstrated significantly more fixations. 


Figure 6 
Average number of fixations for each AOI, task, and group of participants 
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In the next step, the distribution of fixations at particular AOls in time was visualized. For this type of analysis, 
a sequence chart was created. Task 2B was selected as an example (Figure 7). This type of data visualization shows 
how often participants visited each AOI and how much time they spent there. Participants who read the task 
several times (P13, P16) or ended their eye-movement trajectory in an AOI task (P38, P40) could also be identified. 
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Figure 7 


Sequence chart for task 2B 
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Verifying Allocation to Groups 


The ScanGraph tool was used for the next step of data analysis. The tool typically displays groups of partici- 


pants as a graph, though for the present study, the tool was used in a different manner. One of its options allows 
the similarities between participants to be exported as a CSV file. Each group was calculated separately for the 
average similarity between members of the group. The participants with the lowest value of similarity indicated 
the greatest difference. These results revealed five potential candidates (indicated in grey in Figure 8) who were not 
assigned correctly to groups. It would be appropriate to focus on these students and learn whether they genuinely 
belonged to their allocated group. The average similarity of these students to the remaining participants was the 
least. Similarity with other participants is indicated with circular symbols in Figure 8, these students being P16 from 
the MINUS group, P11 and P30 from the AVERAGE group, and P13 and P22 from the PLUS group. 
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Figure 8 


Similarity in the groups of participants 


Ove‘’o 


722@6 


7@OS & sr¢0 
s@9 @@ é¢'0 





5 9909298085 099990095O492) zr 
532392580299 2S80S20928892 S268 
5 39092850 SE9289000S8899 49:28 
5 395995259 50959525099 9922110 
5 90992254059 09292592 9920110 
532692200088 9S888008 892850 
> 999999959009909O 2902990 
3130000890809 09808 S0OOOHOC Eee 
00008005008 S8008 20008080 z¢'0 
§3239252029802888 £09222 829 080 
§ 9959252599098 2£50299909 2200 
§ 9299282508980 2608922822492 
522820000889 09420909 9SS29 £20 
= 9999259598 990925O2%00990 Lr 
| 9990SS2S808 0299900029229 4 0200 
= 32526209 2922920990892 S241Ir0 
SO OOO088) 80000 0OS8e0908000 eee 
= 3952288 9029085099082 S8S 860 
5 @S989 2924208245289 22828445 rt 
elelele) Nellelolel: lolololol@l lolol: lolololttx 
S890 2225209299959 9999992I00 
509 0292029929249 2929291II00 
58 9922490299 24940598 209949 tIr0 
5) 909502529929 9252929242249010' 


ornrondox#HXnnWadararnrnmnnvooadaoxX&nmn ast O BS 
oo Ortartnrtwt nwt ANN NNN MOMONMONH HH /H 
Oaaaqaaqg’aiaaqaaqaqaqaeaaaaqaqaeaaqaedqaaqadaaa a 








P38 


am 
oO Oo ©O 
aad a 


= 
w 
= 


09€‘0 


53339098009 


10992990208 92680 
109029059 @@ L010 
183440084 946 200 





599008 09099 2:60 
~9SS20 8090499 26’ 





5! 196 90082269 © ose'0 


© 8028906996 zz 
2) ©020909220064 «£0 


t+ Nore © 
a NNN NN 
a aaa a. 


P17 
P39 


N ON 
oo = 
aad a 


= 
w 
= 


311 


https://doi.org/10.33225/jbse/20.19.298 


Journal of Baltic Science Education, Vol. 19, No. 2, 2020 


STUDENTS’ ABILITY TO WORK WITH GRAPHS IN PHYSICS STUDIES RELATED TO THREE ISSN 1648-3898 /Print/ 
TYPIGAL STUDENT GROUPS 
(P, 298-316) ISSN 2538-7 138 /oniines 


The next step in the eye-movement data analysis incorporated the data of each of these five participants. 
Students P11 and P30 from the AVERAGE group showed the least values of similarity with others in this group. 
Figure 7 indicates that participant P11 (female), especially, used a completely different sequence of visited AOls 
in task 2B. This participant almost completely omitted the task’s text and concentrated directly on the graphs. The 
data suggest that the students were allocated to the AVERAGE group in the correct manner, but they probably 
cannot sufficiently read graphs. However, the other question of the relationship between knowledge and its use 
in reading graphs then arises. It would also be worth determining whether transferring student P11 to the MINUS 
group would be more appropriate. 

Participant P22 (female) was allocated to the PLUS group. Her results (3 correct and 4 incorrect answers) 
corresponded more to the AVERAGE group (Table 1). The results from ScanGraph (Figure 8) could have been af- 
fected by no AOI visit being recorded for this participant in task 3B. Participant P22’s inclusion in the PLUS group, 
therefore, should be reviewed. 

Participant P13 (male) was allocated to the PLUS group. He had the best overall result in the PLUS group, 
achieving four correct answers and three incorrect answers (Table 1). This participant also demonstrated in task 
2B (Figure 7) one of the greatest numbers of fixations observed in all the stimuli of the experiment. 

A different conclusion applies to participant P16 (female), who was allocated to the MINUS group but obtained 
the best results of all the participants. After the eye-movement data analysis, the teacher who allocated students 
into the groups was shown the results. The teacher stated that the girl suffered from partial deafness, which may 
have led to her being allocated to the MINUS group. Because the tasks were presented on a screen, she was not 
limited by her medical condition during the experiment and achieved one of the best results. 


e Hypothesis 1: This hypothesis was not proved, because task 1A demonstrated the exact opposite, i.e. 
the MINUS group needed the least amount of time of all three groups (although the results were not 
Statistically significant). The MINUS group also demonstrated a time comparable to the other groups 
in tasks 2A, 2B, and 3B (Figure 4). 

e Hypothesis 2: This hypothesis was not proved. The PLUS group achieved the worst results, their ac- 
curacy being only 32.5%. 

e Hypothesis 3: This hypothesis was not proved. Tasks 1B and 2B showed the least number of fixations 
recorded in AOI“C’", which was the correct answer in both cases. In task 3B, both options were correct 
(A and B), and students fixated on them extensively. Students fixated on other answers only minimally 
(Figure 6). 

e Hypothesis 4: This hypothesis was proved. The results from ScanGraph indicated certain individuals who 
behaved differently to the remainder of their group. These were students P11, P13, P16, P22, and P30. 

e Hypothesis 5: This hypothesis was proved, shown in the results in Table 1. In the MINUS group, partici- 
pant P16 demonstrated results that were appropriate to the level of the PLUS group. In the PLUS group, 
participant P13 demonstrated results that were appropriate to the AVERAGE group. 


Comparing hypotheses H1 and H3 and research questions No. 2 and 3 provides enough information to reveal 
similar segments in the obtained data. Clusters showing similar attitudes between the groups of participants in 
solving the given tasks can be observed. The notion that the time required by the participants of a given group to 
solve certain tasks depends on their specific group category is not valid. 

It is also not valid to conclude that the correctness of answers increased as the number of fixations in AOls 
increased (Figure 6). From the point of view of didactics, this result is very interesting, because there were fewer 
fixations for correct answers than incorrect answers (Figures 5 and 6). This was probably due to two factors, one 
being the specific type of graph involved. Few fixations were recorded for graphs which teachers would use more 
frequently than others to teach the topic. The fewest fixations in AOls with correct answers were observed with 
graphs for mechanics (kinematics) and molecular and thermal physics, which are taught in the first and second 
grades (tasks 1C and 2B). In higher grades, other subjects may interest students more. The interest in physics may 
also therefore decline, and students may exhibit less concrete knowledge and be unable to make decisions quickly 
and correctly. 
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Discussion 


Similar research has been conducted previously. Some studies analyse the use of graphs in physics. McDermott 
et al. (1987), for example, analysed student's difficulties in connecting graphs with concepts in physics. Some studies 
have combined this type of analysis with eye-tracking technology. Kekule (2014), for example, analysed students’ 
strategies in examining kinematics graphs. Strobel et al. (2018) used eye-tracking methods for task-irrelevant data 
to impair the processing of graph reading tasks. Many other authors have investigated graph reading in fields other 
than physics. Wainer (1992) studied the comprehension of graphs and tables in the field of geography. Vila and 
Gomez (2016) extracted business information from graphs, etc. However, to the best of our knowledge, no previous 
eye-tracking research has been conducted with participants allocated to groups according to their abilities. The 
teacher in the present experiment allocated students into three groups: PLUS, AVERAGE and MINUS. 

The results of the eye-tracking experiment demonstrated that this type of division may be problematic. A 
comparison of hypotheses H2, H4, and H5 and research question No. 3 indicates that the teacher could not ap- 
propriately allocate students into groups according to their cognitive abilities. The finding then leads to the recom- 
mendation that the information available to teachers about the characteristics displayed by students in certain 
groups, particularly gifted students, should be improved. The myth that the hardworking and “learned” student is 
gifted is still perpetuated by teachers, which is certainly disproven in theory and practice. Comparing the connec- 
tion between hypothesis H5 and research question No. 1 suggests the possibility of recovering students who are 
incorrectly allocated to a group. Hypothesis H5 and research question No. 1 correlate. For example, participants P24 
and P39 (Table 1) in the PLUS group supplied one correct answer and six incorrect answers. These results indicate 
that the teacher's knowledge for allocating gifted students appropriately may have been insufficient. 

During the data analysis, the students demonstrated some interesting behaviour, for which we attempted to 
find an explanation. The first issue concerned students from the PLUS group spending more time at AOI“B” than the 
AVERAGE group. This can be seen in the sequence chart in Figure 7. One possible explanation is that this may have 
been caused by the shape of the curve itself, which could have been appealing to participants in the PLUS group 
and encouraged thinking about variations in the represented process, which is often typical in gifted individuals. 

During the analysis of the recorded data, five participants who were inappropriately assigned to their group 
were identified. Participant P13 performed the most fixations in all AOls, as he continuously sought correct answers. 
This behaviour might imply the effort, care (perhaps pedantic) and responsibility of the student to obtain the cor- 
rect result. This behaviour could have led the teacher to allocating him to the PLUS group. 

By contrast, participant P16, who was allocated to the MINUS group, had the best results of all participants. 
This student is partially deaf, which may adversely affect her abilities during class and may have led the teacher to 
allocating her to the MINUS group. During the eye-tracking experiment, however, she did not need to use hearing 
at all, and she obtained very good results. Another explanation is that she may be a gifted student and belong— 
according to (Betts & Neihart, 1988) typology—to the group of “underground gifted.” Given that this student was 
female, it is a plausible explanation. 

The research studied graph reading in physics. The research offers a comparison of real procedures in read- 
ing graphs by individual students. In cooperation with a physics teacher, the graph reading strategies of students 
allocated to PLUS, AVERAGE and MINUS groups were mapped. 


Conclusions 


The research explored the strategies of students as they solved physics tasks with graphs. The research was 
conducted on a final sample of 40 grammar school students, who were allocated by their teacher into three groups 
(PLUS, AVERAGE and MINUS) according to the teacher's observations and the students’ previous study results in 
physics. 

The results of the eye-tracking research confirmed that the teacher did not always allocate students cor- 
rectly into the groups according to their cognitive abilities. In this context, the results also revealed that success in 
reading graphs in physics is not fundamentally related to allocating students to particular groups. Furthermore, 
the experiment exposed that the allocation of students into the PLUS group was not related to the time students 
needed to complete the tasks. Analogically, the experiment revealed that the number of errors made by students 
from the PLUS group did not relate to their allocation to that group. 
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The research proved that the results of eye-tracking analysis highlighted the students that were not assigned 
to an appropriate group by the teacher. 

The recommendations based on the research follow two directions. The research confirmed that students 
could not rely on their ability to work with graphs, which implies that teachers should practice these skills more 
with students during lessons and employ more graph reading practice in physics education. The second recom- 
mendation is that teachers should educate themselves more about talented and gifted students. 
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